Validate bounded-query sandbox runtime matrix by lpcox · Pull Request #6780 · github/gh-aw-firewall

lpcox · 2026-07-31T03:00:47Z

Summary

add table-driven conformance for all 9 primary-agent/query-runtime combinations
fail unavailable primary/query capabilities before staging and keep all sbx-query combinations security-blocked
add exact-field runtime telemetry, capability reporting, real Docker smoke cleanup, and opt-in gVisor isolation coverage
document support, troubleshooting, one-sandbox-per-query guarantees, and sbx promotion criteria

Current matrix

supported when locally available: Docker or gVisor queries under Docker, gVisor, or sbx primary agents
blocked: every sbx query combination on Docker Sandboxes v0.37.1
no runtime fallback

Validation

build, type-check, lint (0 errors), and 4,937 unit tests passed
real Docker bounded-query isolation passed and left no containers
generated config schemas remained synchronized
full integration suite executed: 74 passed; remaining tests were blocked by local chown/sudo host permissions

Stacked on #6764; merge after #6762, #6763, and #6764.

Add fail-closed 3x3 runtime conformance, safe telemetry, capability reporting, opt-in gVisor coverage, and promotion documentation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7fe22290-7ea3-4de1-be61-849d0c06c958

github-actions · 2026-07-31T03:01:11Z

Documentation Preview

Documentation build failed for this PR. View logs.

Built from commit 624c64c

Copilot

Pull request overview

Adds bounded-query runtime-matrix validation, capability preflights, telemetry, and expanded isolation coverage.

Changes:

Models and reports all nine primary/query runtime combinations.
Adds fail-closed preflights, telemetry, and cleanup validation.
Documents runtime support, troubleshooting, and sbx promotion criteria.

Show a summary per file

File	Description
`.github/workflows/test-gvisor-compat.yml`	Adds gVisor isolation CI.
`containers/bounded-query/broker/broker.js`	Emits query telemetry.
`containers/bounded-query/broker/config.js`	Loads primary backend.
`containers/bounded-query/broker/runtime-telemetry.js`	Persists restricted telemetry records.
`containers/bounded-query/broker/server.js`	Wires lifecycle telemetry.
`docs/awf-config-spec.md`	Specifies matrix and promotion requirements.
`docs/bounded-queries.md`	Documents matrix and troubleshooting.
`docs/sbx-integration.md`	Clarifies sbx support criteria.
`scripts/ci/report-bounded-query-runtime-matrix.js`	Generates capability reports.
`scripts/ci/report-bounded-query-runtime-matrix.test.ts`	Tests matrix reporting.
`scripts/ci/smoke-bounded-queries.sh`	Requires Docker matrix support.
`src/bounded-query/manager.ts`	Adds runtime preflight orchestration.
`src/bounded-query/manager.test.ts`	Tests preflight ordering.
`src/bounded-query/mount-policy.test.ts`	Makes real-path assertion portable.
`src/bounded-query/preflight.ts`	Adds primary/query availability checks.
`src/bounded-query/preflight.test.ts`	Covers capability failures.
`src/bounded-query/runtime-matrix.ts`	Defines matrix and telemetry types.
`src/bounded-query/runtime-matrix.test.ts`	Exercises matrix contracts.
`src/bounded-query/wrapper.test.ts`	Extends concurrent-test timeout.
`src/services/bounded-query-service.ts`	Passes primary backend to broker.
`src/services/bounded-query-service.test.ts`	Tests backend environment mapping.
`tests/integration/bounded-query-isolation.test.ts`	Adds gVisor selection and cleanup.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 22/22 changed files
Comments generated: 4
Review effort level: Balanced

github-actions · 2026-07-31T04:19:58Z

✅ Copilot review passed with no inline comments.

@lpcox Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7fe22290-7ea3-4de1-be61-849d0c06c958

github-actions · 2026-07-31T05:02:54Z

✅ Smoke Claude passed

github-actions · 2026-07-31T05:02:56Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-07-31T05:02:56Z

📰 DEVELOPING STORY: Smoke Docker Sbx reports failed. Our correspondents are investigating the incident...

github-actions · 2026-07-31T05:02:59Z

🔌 Smoke Services — All services reachable! ✅

github-actions · 2026-07-31T05:03:00Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-07-31T05:03:05Z

✅ Contribution Check completed successfully!

github-actions · 2026-07-31T05:03:12Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

github-actions · 2026-07-31T05:03:15Z

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

github-actions · 2026-07-31T05:03:19Z

✅ Smoke Gemini completed. All facets verified. 💎

github-actions · 2026-07-31T05:03:23Z

✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

No further action needed - smoke test summary posted as comment

github-actions · 2026-07-31T05:03:24Z

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

github-actions · 2026-07-31T05:03:26Z

✅ Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓

github-actions · 2026-07-31T05:03:28Z

✅ Build Test Suite completed successfully!

github-actions · 2026-07-31T05:03:31Z

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

github-actions · 2026-07-31T05:03:56Z

✅ Security Guard completed successfully!

Security review complete for PR #6780 (Validate bounded-query sandbox runtime matrix). Analysis scope: 4 security-relevant files with 56 new lines. Findings: All security-critical components passed review. No firewall rule changes, no capability additions, no authentication bypasses, no secrets exposure. Input validation is robust (whitelisted backends and categories, regex-enforced category pattern). File permissions restrictive (0o600). Error handling safe. This is a security-neutral observability feature addition. No security concerns to report.

github-actions · 2026-07-31T05:05:54Z

Smoke Test Results (@lpcox)

✅ GitHub MCP connectivity (PR refactor: split bounded query runners #6762, fix: isolate bounded-query private state from agent mounts #6758)
✅ GitHub.com connectivity (HTTP 200)
✅ File write/read test

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:05:57Z

Smoke Test: Claude Engine Validation

Check	Result
API status	✅ PASS
gh check	✅ PASS
file status	✅ PASS

Overall result: PASS

Generated by Smoke Claude for #6780 · haiku45 · 31.7 AIC · ⊞ 3.6K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:06:25Z

Thanks — the PR description is clear and it references related work, which matches the PR-process guidance in CONTRIBUTING.md.

One thing I couldn't verify from the changed files here: new/updated tests for the bounded-query matrix and documentation updates for the new runtime/telemetry behavior. CONTRIBUTING.md asks contributors to "write tests for new functionality" and to "update documentation for any new features." If those changes are elsewhere, please ignore this note; otherwise, please add them so the PR fully meets the contribution guidelines.

Generated by Contribution Check for #6780 · gpt54mini · 2.5 AIC · ⊞ 24.8K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:06:30Z

Smoke Test: Services Connectivity — FAIL

Redis: ❌ Temporary failure in name resolution (host.docker.internal)
PostgreSQL pg_isready: ❌ no response
PostgreSQL SELECT 1: ❌ could not translate host name

Overall: FAIL — host.docker.internal did not resolve in the sandbox; no connectivity to host service containers.

🔌 Service connectivity validated by Smoke Services
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:06:43Z

Smoke Test: Copilot BYOK (Direct) ✅ PASS

Running in direct BYOK mode via COPILOT_PROVIDER_API_KEY → api-proxy sidecar → api.githubcopilot.com

✅ GitHub MCP connectivity (read PR data)
✅ GitHub.com accessibility (HTTP 200)
✅ File I/O (write/read confirmed)
✅ BYOK inference (responding to prompts)

All tests passed. @lpcox

🔑 BYOK report filed by Smoke Copilot BYOK
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:06:52Z

Recent merged PRs:

refactor: split bounded query runners
fix: isolate bounded-query private state from agent mounts

Checks:

PR query: ✅
GitHub title: ✅
File write/read: ✅
Build: ✅

Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:07:08Z

Smoke Test: API Proxy OTEL Tracing

✅ Module Loading: otel.js loads, exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled, plus internal exporter helpers.
✅ Test Suite: otel.test.js + otel-fanout.test.js — 59/59 passed.
✅ Env Var Forwarding: src/services/api-proxy-env-config.ts buildOtelEnv() forwards OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS, GITHUB_AW_OTEL_TRACE_ID, GITHUB_AW_OTEL_PARENT_SPAN_ID, and sets OTEL_SERVICE_NAME.
✅ Token Tracker Integration: onUsage callback present in token-tracker-http.js (line 348/406) as the OTEL hook point, invoked after normalized usage extraction.
✅ OTEL Diagnostics: No errors during local module/test validation; graceful degradation confirmed (isEnabled/no-endpoint path).

All scenarios pass. No issues found.

📡 OTel tracing validated by Smoke OTel Tracing
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:07:21Z

@lpcox Smoke test (BYOK AOAI Entra):
• GitHub MCP: ✅
• HTTP: ✅
• File I/O: ✅
• BYOK inference: ✅
Mode: AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL via api-proxy → Azure OpenAI (o4-mini-aw) authenticated via Microsoft Entra
Overall: PASS

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:07:38Z

Chroot Version Comparison Test Results

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.13	Python 3.12.13	✅ YES
Node.js	v24.18.0	v22.23.1	❌ NO
Go	go1.22.12	go1.22.12	✅ YES

Overall: FAILED — Node.js version mismatch between host and chroot environments. smoke-chroot label not added since not all versions matched.

Tested by Smoke Chroot
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:07:44Z

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (api.github.com) reachable: allowed=200
✅ Blocked domain (example.com) denied: OK: example.com was blocked (CONNECT 403)

Overall status: PASS

@lpcox

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:08:15Z

Gemini Smoke Test Results

GitHub MCP Testing: ❌ (Access Denied by policy)
GitHub.com Connectivity: ❌ (Status 000, Exit Code 7)
File Writing Testing: ✅
Bash Tool Testing: ✅

Overall Status: FAIL

Note: PR titles for #6762 and #6758 were inaccessible due to secrecy policy.

💎 Faceted by Smoke Gemini
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:10:13Z

@lpcox smoke-copilot-byok results:

GitHub MCP Testing: ❌ (integrity policy)
GitHub.com Connectivity: ✅
File I/O: ✅
BYOK Inference: ✅
Overall: FAIL
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)
_{Add label ready-for-aw to run again}

github-actions · 2026-07-31T05:10:38Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A (ran OK)	✅ PASS
.NET	json-parse	✅	N/A (ran OK)	✅ PASS
Go	color	✅	ok	✅ PASS
Go	env	✅	ok	✅ PASS
Go	uuid	✅	ok	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	all passed	✅ PASS
Node.js	execa	✅	all passed	✅ PASS
Node.js	p-limit	✅	all passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Notes: Java Maven builds required -Dmaven.repo.local override since ~/.m2 was root-owned/read-only in this sandbox (not a firewall issue); proxy settings in ~/.m2/settings.xml were applied as instructed.

Generated by Build Test Suite for #6780 · aut00 · 65.5 AIC · ⊞ 11.3K · ◷
_{Add label ready-for-aw to run again}

Copilot AI review requested due to automatic review settings July 31, 2026 03:00

Copilot started reviewing on behalf of lpcox July 31, 2026 03:01 View session

Copilot AI reviewed Jul 31, 2026

View reviewed changes

Comment thread scripts/ci/report-bounded-query-runtime-matrix.js Outdated

Comment thread src/bounded-query/preflight.ts Outdated

Comment thread src/bounded-query/manager.ts

Comment thread containers/bounded-query/broker/runtime-telemetry.js

fix: address runtime matrix review feedback

cf0ab53

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7fe22290-7ea3-4de1-be61-849d0c06c958

lpcox added the ready-for-aw label Jul 31, 2026

lpcox temporarily deployed to aoai-model July 31, 2026 05:03 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot label Jul 31, 2026

github-actions Bot added the smoke-claude label Jul 31, 2026

github-actions Bot added the smoke-copilot-byok label Jul 31, 2026

github-actions Bot added the smoke-codex label Jul 31, 2026

lpcox temporarily deployed to aoai-model July 31, 2026 05:07 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-byok-aoai-entra label Jul 31, 2026

lpcox temporarily deployed to aoai-model July 31, 2026 05:07 — with GitHub Actions Inactive

github-actions Bot mentioned this pull request Jul 31, 2026

[aw] No-Op Runs #6709

Open

github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Jul 31, 2026

github-actions Bot added the build-test label Jul 31, 2026

lpcox merged commit 56d38d5 into lpcox-add-sbx-query-runner Jul 31, 2026
113 of 115 checks passed

lpcox deleted the lpcox-validate-sandbox-matrix branch July 31, 2026 05:20

Uh oh!

Conversation

lpcox commented Jul 31, 2026

Summary

Current matrix

Validation

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation Preview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 31, 2026

Uh oh!

github-actions Bot commented Jul 31, 2026

Smoke Test: Claude Engine Validation

Uh oh!

github-actions Bot commented Jul 31, 2026

Uh oh!

github-actions Bot commented Jul 31, 2026

Uh oh!

github-actions Bot commented Jul 31, 2026

Smoke Test: Copilot BYOK (Direct) ✅ PASS

Uh oh!

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading

github-actions Bot commented Jul 31, 2026 •

edited

Loading